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Abstract 



Most specifications of garbage collectors concentrate on the low-level algorithmic details of how to 
find and preserve accessible objects. Often, they focus on bit-level manipulations such as "scanning 
stack frames," "marking objects," "tagging data," etc. While these details are important in some 
contexts, they often obscure the more fundamental aspects of memory management: what objects 
are garbage and why? 

We develop a series of calculi that are just low-level enough that we can express allocation 
and garbage collection, yet are sufficiently abstract that we may formally prove the correctness of 
various memory management strategies. By making the heap of a program syntactically apparent, 
we can specify memory actions as rewriting rules that allocate values on the heap and automatically 
dereference pointers to such objects when needed. This formulation permits the specification of 
garbage collection as a relation that removes portions of the heap without affecting the outcome of 
the evaluation. 

Our high-level approach allows us to compactly specify and prove correct a wide variety of 
memory management techniques, including standard trace-based garbage collectors (i.e., the family 
of copying and mark/sweep collectors), generational collection, and type-based tag-free collection. 
Furthermore, since the definition of garbage is based on the semantics of the underlying language 
instead of the conservative approximation of inaccessibility, we are able to specify and formally 
prove the idea that type inference can be used to collect some objects that are accessible but never 
used. 



1 Memory Safety 



Advanced programming languages manage memory allocation and deallocation automatically. Au- 
tomatic memory managers, or garbage collectors, significantly facilitate the programming process 
because programmers can rely on the language implementation for the delicate tasks of finding and 
freeing unneeded objects. Indeed, the presence of a garbage collector ensures memory safety in the 
same way that a type system guarantees type safety: no program in an advanced programming lan- 
guage will crash due to dangling pointer problems while pointer allocation, access, and deallocation 
is transparent. However, in contrast to type systems, memory management strategies and particu- 
larly garbage collectors rarely come with a compact formulation and a formal proof of soundness. 
Indeed, since garbage collectors work on the machine representations of abstract values, the very 
idea of providing a proof of memory safety sounds unrealistic given the lack of simple models of 
memory operations. 

The recently developed syntactic approaches to the specification of language semantics by 
Felleisen and Hieb [11] and Mason and Talcott [23, 24] are the first execution models that are 
intensional enough to permit the specification of memory management actions and yet are suffi- 
ciently abstract to permit compact proofs of important properties. Starting from the A„-S calculus 
of Felleisen and Hieb, we design compact specifications of a number of memory management ideas 
and prove several correctness theorems. 

The basic idea underlying the development of our garbage collection calculi is the representation 
of a program's run-time memory as a global series of syntactic declarations. The program evaluation 
rules allocate large objects in the global declaration, which represents the heap, and automatically 
dereference pointers to such objects when needed. As a result, garbage collection can be specified 
as any relation that removes portions of the current heap without affecting the result of a program's 
execution. 

In Section 2, we present a small functional programming language, Age, with a rewriting seman- 
tics that makes allocation explicit. In Section 3, we define a semantic notion of garbage collection 
for Age and show that there is no optimal collection strategy that is computable. In Section 4, we 
specify the "free-variable" garbage collection rule which models tracing-based collectors including 
mark/sweep and copying collectors. We prove that the free- variable rule is correct and provide two 
"implementations" at the syntactic level: the first corresponds to a copying collector, the second 
to a generational one. In the context of generational collection, we also illustrate how mutable 
reference cells render the original generational collector unsound and how the problem can be fixed 
at an abstract level. 

In Section 5, we show how the use of abstract syntax facilitated our work in the first few sections. 
Specifically, the abstract syntax of Age implicitly carries the shape (size and pointer information) 
of allocated objects, which permits specifications of GC algorithms that ignore the problem of 
traversing the memory graph in the absence of guiding information. To illustrate how to refine 
Age just enough to address this problem, we formalize so-called "tag-free" collection algorithms 
for explicitly- typed, monomorphic languages such as Pascal and Algol [7, 36, 8]. We show how to 
reconstruct necessary shape information about values from types during garbage collection. We 
are able to prove the correctness of the garbage collection algorithm by using a well known type 
preservation argument. We then sketch how the type- based garbage collection algorithms can be 
extended to a language with a polymorphic type system in the style of the Girard-Reynold's System 
F [13, 14, 30]. Our formulation leads to a fairly simple proof of the correctness of Tolmach's garbage 
collector for the Gallium ML compiler [34]. 

In Section 6, we justify our semantic definition of garbage by showing that Milner-style type 
inference can be used to prove that an object is semantically garbage even though the object is still 
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reachable. While previous authors have sketched this idea [6, 3, 16, 12], we are the first to present 
a formal proof of this result. The proof is obtained by casting the well-known interpretation of 
types as logical relations into our framework. 

Section 7 discusses related work and Section 8 closes with a summary. 



Programs: 



(variables) x,y,z £ Var 



(integers) 

(expressions) 

(heap values) 

(heaps) 

(programs) 

(answers) 



i 
e 
h 
H 
P 
A 



G 
G 

e 

G 
G 
G 



Int 
Exp 
Hval 
Heap 
Prog 
Ans 



■■■ | -2 | -1 |0 ! 1 I 2 | ... 

x | i | (ei, e 2 ) | 7Ti e | 7r 2 e | Xx.e | e x e 2 

i I (£1,2:2) I Xx.e 

{xi = hx,...,x n = h n } 

letrec H in e 

letrec i7 in x 



Evaluation Contexts and Instruction Expressions: 

(contexts) E G Ctxt ::= [ ] ] (E, e) \ (x, E) \ n, E \ E e \ x E 
(instructions) / G Instr ::= h \ x \ x y 



Figure 1: The Syntax of Age 



2 Modeling Allocation: Age 

Syntax: The syntax of Age (see Figure 1) is that of a conventional, higher-order, applicative 
programming language based on the A-calculus. Following the tradition of functional programming, 
a Age program (P) consists of some mutually recursive definitions (H) and an expression (e). The 
global definitions are useful for defining mutually recursive procedures, but their primary purpose 
here is to represent the run-time heap of a program. In general, there can be cycles in a heap so 
we use letrec instead of let as the binding form. Expressions are either variables (x), integers (i), 
pairs ((€1,62)), projections (7^), abstractions (Ax.e), or applications (ei €2)- 



FV(i) = 0 

FV(x) = {x} 

FV((e l ,e 2 }) = FV( ei )UFV(e 2 ) 

FV{-Ki e) = FV{e) 

FV(Xx.e) = FV(e) \ {x} 

FV{e l e 2 ) = FV{e\) U FV(e 2 ) 

FV({x 1 = h 1 ,...,x n = h n }) = (FV(h 1 )U...UFV(h n ))\{x u ...,x n } 
FV(\etrec H in e) = (FV(H) U FV(e)) \ Dom(H) 

Figure 2: Calculating Free Variables 



Formally, the heap is a series of pairs, called bindings, consisting of variables and heap values. 
Heap values are a semantically significant subset of expressions. The order of the bindings is 
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irrelevant. In addition, each variable must be bound to at most one heap value in a heap. Hence, 
we treat heaps as sets and, when convenient, as finite functions. We write Dom(H) to denote the 
bound variables of H and Rng(H) to denote the heap values bound in H. We sometimes refer to 
variables bound in the heap as locations or pointers. 

The language contains two binding constructs: Xx.e binds x in e and letrec H in e binds the 
variables in Dom(H) to the expressions Rng(H) in both the values in Rng(H) and in e. Following 
convention, we consider expressions to be equivalent up to a consistent a-conversion of bound 
variables. The free variables of expressions, heaps, and programs are determined by the definitions 
of Figure 2. 

Considering programs equivalent modulo a-conversion and the treatment of heaps as sets instead 
of sequences hides many of the complexities of memory management. In particular, programs are 
automatically considered equivalent if the heap is re-arranged and locations are re-named as long 
as the "graph" of the program is preserved. This abstraction allows us to focus on the issues 
of determining what bindings in the heap are garbage without specifying how such bindings are 
represented in a real machine. 

Mathematical Notation: We use X l+J X' to denote the union of two disjoint sets, X and X'. 
We use H l±l H' to denote the union of two heaps whose domains are disjoint. We use {ei/x}e 2 to 
denote capture-avoiding substitution of the expression ei for the free variable x in the expression 
e 2 . We use X \ X' to denote {x € X \ x <£ X'}. 

Semantics: The rewriting semantics for Age is an adaptation of the standard reduction function 
of the A„-S calculus [11]. Roughly speaking, this kind of semantics describes an abstract machine 
whose states are programs and whose instructions are relations between programs. The desired 
final state of this abstract machine is an answer program (A) whose body is a pointer to some 
value, such as an integer, in the heap. 

Each rewriting step of a program letrec H in e proceeds according to a simple algorithm. If 
the body of the program, e, is not a variable, it is partitioned into an evaluation context E (an 
expression with a hole [ ] in the place of a sub-expression), which represents the control state, 
and an instruction expression /, which roughly corresponds to a program counter: e = EU1- The 
instruction expression determines the next expression e' and any changes to the heap resulting in 
a new heap H' . Putting the pieces together yields the next program in the evaluation sequence: 
letrec H' in ELe'l. Each instruction determines one transition rule of the abstract machine. For- 
mally, a rule denotes a relation between programs. A set of rules denotes the union of the respective 
relations. 

We use the following conventions: Let G be a set of program relations, and let P and P' be 
programs. Then, 

P h-^- P' means P rewrites to P' according to one of the rules in G and i-^-* is the reflexive, 
transitive closure of i-^-k 

G + r is the union of G with the rule r. 

A program P is canonical with respect to G iff there is no rule in G and no P' such that 
P^P'. 

P Jj-G P' means that P i-^*P' and P' is canonical with respect to G. 

P ftG means that there exists an infinite sequence of programs Pi such that P i-^ P\ 
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Figure 1 defines the set of evaluation contexts and instruction expressions for Age. The definition 
of evaluation contexts (E) reflect the left-to-right evaluation order of the language. All terms to 
the left of the path from the root to the hole are variables; the terms on the right are arbitrary. 
Instruction expressions (7) consist of heap values (h), applications of (pointers to heap- allocated) 
procedures to (pointers to heap-allocated) values, and projections of (pointers to heap-allocated) 
tuples. 



(alloc) letrec H in EUH ^ letrec H l±l {x = h} in EZxl 

(proj) letrec H in El^i ar] letrec H in ELxil [H{x) = <xi, x 2 ) and i = 1, 2) 

(app) letrec H in Elx yl ^ letrec H W {z = H{y)} in ELel (H(x) = Xz.e) 

Figure 3: Rewriting Rules for Age 



The evaluation rules for Age (see Figure 3) reflect the intentions behind our choice of instruction 
expressions. The transition rule alloc models the allocation of values in the heap by binding the 
value to a new variable and using this variable in its place in the program. Note that the "l+l" 
notation carries the implicit requirement that the newly allocated variable x cannot be in the 
domain of the heap H . The transition proj specifies how a projection instruction extracts the 
appropriate component from a pointer to a heap-allocated pair. Similarly, app is a transliteration 
of the conventional /3-value rule into our modified setting. It binds the formal parameter of a 
heap-allocated procedure to the value of the pointer given as the actual argument, and places the 
expression part of the procedure into the evaluation context. Multiple applications of the same 
procedure require cv-conversion to ensure that the formal parameter does not conflict with bindings 
already in the heap 1 . We use R to abbreviate the union of the rules alloc, proj, and app. 

As a simple example of evaluation under R, consider rewriting the program P — letrec {x = 
1} in (Xy.Tti y)(x,x): 

l,z= Xy.TTy y} in z (x,x) 
1, z = Xy.TTi y, w = (x, x)} in z w 
1,2 = Xy.Kx y,w = (x,x),y = (x, x)} in 7r : y 
l,z= Xy.ir 1 y,w = (x,x),y = (x, x)} in x 

We can break P into an evaluation context Eq = [ ] {x, x) and instruction Iq = Ay. 7^ y. Since 7o is a 
heap value, alloc applies. We choose a new variable z, bind the heap value to z, and replace it with z 
in the hole of £x>, resulting in the program letrec {x = 1, z = Ay. 7^ y} in z (x, x). This program can 
be broken into evaluation context E\ = z [ ] and instruction Ii — (x. x). Ii is also a heap value, so 
alloc applies, we choose a new variable w, bind the heap value to w, and replace the instruction with 
w, resulting in the program letrec {x — l,z = Ay.^ y,w = (x,x)} in z w. This program can be 
broken into an empty context (E? = C ]) and instruction I2 = z w. Since z is bound to a procedure 
and w is bound in the heap, app applies. The heap value that w is bound to ((x, a;)) is bound to 
the formal argument of the procedure (y) in the heap and the body of the procedure replaces the 
instruction, resulting in the program letrec {x = 1, z = Ay.^ y,w = (x, x), y = (x, x)} in iti y. This 
program can be broken into an empty evaluation context (E3 = [ ]) and instruction ^3 = 7Ti y. 
Since y is bound to a pair, the proj rule applies and the instruction is replaced with the first 

J An alternative rule for application substitutes the actual argument (y) for the formal (z) within e and performs 
no allocation. This rule is essentially equivalent to app, but the definition above simplifies the proofs of Section 6. 



letrec {x = 1} in (Ay. 7^ y)(x 1 x) A-^- letrec {x — 

alloc 1 . t 

1 — > letrec {x = 
^» letrec {x ' 
letrec {x = 
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component of the pair, namely x, resulting in the answer program letrec {x = 1, z = \y.K\ y,w = 
(x, x). y = (x, x)} in x. 

The canonical programs of Age are either answers or stuck programs. The latter correspond 
to machine states that result from the misapplication of primitive program operations or unbound 
variables. 

Definition 2.1 (Stuck Programs) A program is stuck if it is of one of the following forms: 

• letrec H in E Dr; x] (x £ Dom(H) or H(x) ^ (x 1 ,x 2 )) or 

• letrec H in E[x yl (x or y £ Dom(H) or H(x) ^ Xz.e). 

All programs either diverge or evaluate to an answer or a stuck program. Put differently, the 
evaluation process defines a partial function from Age programs to canonical programs [11, 37]. 

Theorem 2.2 If P is a closed program, then there exists at most one P' such that P $r P' . 

Proof (sketch): The theorem relies on the following lemma which shows that at each evaluation 
step, the instruction expression is uniquely determined. 

Lemma 2.3 (Unique Decomposition) If P — letrec H in e is a Xgc program, then either P is 
stuck, P is an answer, or else there exists a unique evaluation context E and instruction I such 
that e = EIH. 

Proof (sketch): We show by induction on the structure of e that either e is a variable or there 
exists a unique E and /, such that e = EU1 . Here, we give one case of the induction as an example: 
suppose e = (ei e 2 ). There are four cases to consider. If e\ is not a variable, then by induction, 
there exists an E', I, such that e x - E'Ul, in which case we take E - E' e 2 . If e\ is a variable, 
then ei = x for some x and H(x) = Xz.e' else e would be stuck. Suppose e 2 is not a variable. Then 
by induction, e 2 = E'Ul and we can take E = x E'. Otherwise, e 2 = y and we can take E = [ ] 
and / = (ci e 2 ). Other cases follow similarly. D 

The rest follows from the lemma in a straightforward manner, given that P is closed. □ 

3 A Semantic Definition of Garbage 

Since the semantics of Age makes the allocation of values explicit, including the implicit pointer 
dereferencing in the language, we can also define what it means to garbage collect a value in the 
heap and then analyze some basic properties. A binding x = h in the heap of a program is garbage 
if removing the binding has no "observable" effect on running the program. In our case, we consider 
only integer results and non-termination to be observable. 

Definition 3.1 (Kleene Equivalence) (Pi,Gi) ~ (P 2 ,G 2 ) means Pi ij- Gl letrec H x in x where 
Hi(x) = i if and only if P 2 Jk? 2 letrec H 2 in y and H 2 {y) = i. If Gi = G 2 = R, then we simply 
write Pi ~ P 2 . 

A binding is garbage if removing it results in a program that is Kleene equivalent to the original 
program: 

Definition 3.2 (Garbage) Given a program P = letrec H l±l {x — h} in e, the binding x = h is 
garbage with respect to P iff P ~ letrec H in e. 
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A collection of a program is the same program with some garbage bindings removed. An optimal 
collection of a program is a program with as many garbage bindings removed as possible. 

Definition 3.3 (Collection, Optimal Collection) The program letrec H in e is a collection of 
P = letrec H l+l H' in e iff all bindings x = h in H' are garbage with respect to P. The program 
letrec H in e is an optimal collection of P iff no binding in H is garbage with respect to P. 

Unfortunately, there can be no optimal garbage collector because determining whether a binding 
is garbage or not is undecidable. 

Proposition 3.4 (Garbage Undecidable) Determining if a binding is garbage in an arbitrary 
closed Xgc program is undecidable. 

Proof: Since Age is essentially an untyped A-calculus, determining whether an arbitrary closed 
program P = letrec H in e halts with an answer or not is undecidable. Assume that GC is an 
algorithm for determining whether or not some binding in the heap is garbage. Then, in particular, 
GC can tell whether or not x = (x^x^) is garbage in the program: 

P' — letrec H l+l {xi = 1, X2 — 2, x = (x\, X2)} in (\y.wi x) e 

where x does not occur in Dom(H). We will show that GC claims that the binding for x is garbage 
if and only if P terminates with an answer, which proves that GC cannot exist. 

Assume GC claims the binding for x is garbage. Then, the program letrec H in (Ay. 7^ x)e is 
Kleene equivalent to P', which clearly implies that P' either diverges or gets stuck, because this 
program will always either diverge or get stuck. To see this, note that evaluation will become stuck 
when x is dereferenced by the projection operation. Given an infinite sequence of reduction steps 
for P', we can easily produce an infinite sequence of reduction steps for P, since this sequence must 
occur while evaluating e. Similarly, given a sequence of reduction steps which takes P' to a stuck 
program, we can produce a sequence of reduction steps that causes P to become stuck, because P' 
must become stuck while evaluating e, since x is still bound in P' . Consequently, if GC claims x is 
garbage, then P either diverges or gets stuck. 

Now assume GC claims that the binding for x is not garbage. This implies the existence of a 
heap value h and variable z such that 

pi | __ > *| etrec jji y _ X2 _ 2^ x = ( a ; 1) X2 ^ z — j n (Ay.7Ti x) z 

Otherwise, x would never be involved in a reduction step. Given this finite reduction sequence for 
P', it follows that P i-^-*letrec H' l±) {z = h} in z In short, the evaluation of P terminates with an 
answer given the assumption that x is not garbage. 

Since the two cases show that GC claims x is not garbage if and only if P terminates with an 
answer, we have established the proposition. □ 

4 Reachability-Based Garbage Collection 

Since computing an optimal collection is undecidable, a garbage collection algorithm must conser- 
vatively approximate the set of garbage bindings. Most garbage collectors compute the reachable 
set of bindings in a program, given the variables in use in the current instruction expression and 
control state. All reachable bindings are preserved; the others are eliminated. 
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In Section 4.1 we present a general specification for reachability-based garbage collection and 
prove that it only collects true garbage. In Section 4.2 we give a high-level description of an 
algorithm based on tracing collection and prove that the algorithm satisfies our specification. In 
Section 4.3 we specialize our algorithm to one that takes advantage of a generational partition and 
in Section 4.4, we show how generational collection works in the presence of assignment. 



4.1 The Free- Variable Rule 

Following Felleisen and Hieb [11], reachability in Age is formalized by considering free variables. 
The following "free-variable" GC rule describes bindings as garbage if there are no references to 
these bindings in the other bindings, nor in the currently evaluating expression: 

(fv) letrec Hi 1+J H 2 in e ^ letrec Hi in e (Dom(H 2 ) D FV{\etrec Hi in e) = 0) 

The fv rule is correct in that it only removes garbage, and thus computes valid collections. The keys 
to the proof of correctness of fv are a postponement lemma and a diamond lemma. The statements 
of these lemmas can be summarized by the diagrams of Figure 4 respectively, where solid arrows 
denote relations that are assumed to exist and dashed arrows denote relations that can be derived 
from the assumed relations. 



fv 



Pi 



fv 



P2 



R 



R 



R 



R 



Ps 



Pi 



f V ' ^ ' 2 f v 

Figure 4: Postponement and Diamond Properties 



1 

P 3 



Lemma 4.1 (Postponement) If Pi P 2 3* P 3 , then there exists a P 2 such that Pi 
P^Ps- 

~ Proof: The proof proceeds by case analysis on the elements of R: 

alloc: Assume letrec H X WH 2 in E[hl ^ letrec Hi in EUA ^ letrec HiW{x = h} in Elx]. We 
only need to show that fv applies after performing the allocation: 

letrec Hi l±l H 2 in E[_K] ^ letrec H x l+J H 2 U {x = h} in Elx] 

But this is clear since allocation only adds x to the locations of the resulting program, and x 
cannot be in Dom(H 2 ) by the side-condition of the alloc rule. Consequently: 

letrec Hi t+J H 2 1+1 {x = h} in A letrec Hi l±) {x = h} in ELxl 
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proj: Assume letrec Hi 1+1 H 2 in Eliv; x] letrec H l in F[7T; x] letrec Hi in F[a:,] Since 
proj applies, x must be bound to some pair {xi,x 2 ) and furthermore, x must be in Hi else 
fv does not apply. Thus we may swap the steps: 

letrec H t l±J H 2 in Fl>; x] ^ letrec Fj 1+) F 2 in EDc,-] letrec Fi in F[>;] 

app: Assume letrec Hi l+J F 2 in Elx y] s-^4 letrec Fx in E[_x y] letrec Hi l+J {z = ft} in F[e] 
Since app applies, x must be bound to Az.e, y must be bound to ft, and z cannot occur in the 
domain of H x . Furthermore, both x and y must be bound in Hi. Performing the application 
first yields: 

letrec H x l+J H 2 in Elx y] ^ letrec Fi W H 2 ttl {z = ft} in F[e] 

The resulting program is well formed only if z is not already bound in H 2 . This can be 
ensured by a-converting the program if necessary. Let x' be a variable that occurs in e. Then 
either x' is z or x' occurs free in Xz.e. In either case, x' is necessarily bound in H\ l+J {z = ft} 
in order for the fv rule to apply, thus x' is not bound in H 2 . Consequently, we can take the 
following step: 

letrec F! W F 2 l+J {z = ft} in Ele] ^ letrec Fj l+J {2 = ft} in E [e] 

□ 

Lemma 4.2 (Diamond) 7/ Pi P 2 and A P 2 , then there exists a P 3 such that P 2 1-^4 P 3 
andP^^P 3 . 

Proof: Assume Fj = letrec H x W F 2 in £"[/], F 2 = letrec H x in F[J] , and P x ^ F 2 . We can 

easily show by case analysis on the elements of R that if Fj p£ where P 2 = letrec /z"i l+J F 2 1+J 
F 3 in Ele] , for some #3 and e, then P 2 P 3 and P 2 i-^4 P 3 where P 3 = letrec Fi t+lF 3 in Ele] . 

□ 

With the Postponement and Diamond Lemmas in hand, it is straightforward to show that fv 
is a correct GC rule. 

Theorem 4.3 (Correctness of fv) If P t-^ P' , then P' is a collection of P. 

Proof: Let F = letrec Hi l+l F 2 in e and F = letrec Hi in e such that F F. We must show 
F evaluates to an integer value iff P' evaluates to the same integer. Suppose P' JJ.^ letrec H in x 
and H(x) = i. By induction on the number of rewriting steps using the Postponement Lemma, 
we can show that P -IJ-jR letrec H i±l F 2 in x and clearly (F 1+) H 2 ){x) = H(x) = i. Now suppose 
F i)-R letrec F in x and F(:c) = i. By induction on the number of rewriting steps using the Diamond 
Lemma, we know that there exists an H' such that P' §r letrec H' in x and letrec H in x 
letrec H' in a;. Thus, x must be bound in H' and since fv only drops bindings, H'(x) = i. □ 

This theorem shows that a single application of fv results in a Kleene equivalent program. A 
real implementation interleaves garbage collection with evaluation. The following theorem shows 
that adding fv to R preserves evaluation. 

Theorem 4.4 For all programs P, (P, R) ~ (P, F + fv). 
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Proof: Clearly any evaluation under R can be simulated by i? + fv simply by not performing any 
fv steps. Thus, if P $r A then P ty R+ f v A. Now suppose P ty R+ f v letrec Hi in x x and Hi{x{) = i. 
Then there exists a finite rewriting sequence using R + fv as follows: 

P ^ Pl «+$ p 2 *t% letrec in x x 

We can show by induction on the number of rewriting steps in this sequence, using the Postponement 
Lemma, that all fv steps can be performed at the end of the evaluation sequence. This provides us 
with an alternative evaluation sequence where all the R steps are performed at the beginning: 

P ^ P [ . . . ,A P ' n ^ Pn+1 ^ Pn+2 ^ . . . ^ letrec H x in x x 

Since fv does not affect the expression part of a program and only removes bindings from the 
heap, P' n = letrec Hi l±J H 2 in x for some H 2 . Thus, P $r letrec Hi l±l H 2 in x. Since Hi(x) = i, 
(Hi®H 2 )(x) = i. □ 

4.2 The Free- Variable Tracing Algorithm 

The free-variable GC rule is a specification of a garbage collection algorithm. It assumes some 
mechanism for partitioning the set of bindings into two disjoint pieces such that one set of bindings 
is unreachable from the second set of bindings and the body of the program. Real garbage collec- 
tion algorithms need a deterministic mechanism for generating this partitioning. It is possible to 
formulate an abstract version of such a mechanism, the free- variable tracing algorithm, by lifting 
the ideas of mark-sweep and copying collectors to the level of program syntax. 

We adopt the terminology of copying collection in the description of the free-variable tracing 
algorithm. We use two heaps and a set: a "from-heap" (#/), a "scan-set" (5), and a "to-heap" 
(H t ). The from-heap is the set of bindings in the current program and the to-heap will become 
the set of bindings preserved by the algorithm. The scan-set records the set of variables reachable 
from the to-heap that have not yet been moved from the from-heap to the to-heap. The scan-set 
is often referred to as the "frontier." 

The body of the algorithm proceeds as follows: A variable x is removed from S such that Hj 
has a binding for x. If no such locations are in 5, the algorithm terminates. Otherwise, it scans the 
heap value h to which x is bound in the from-set Hf, looking for free variables. For each y e FV(h), 
it checks to see if y has already been forwarded to the to-set H t . Only if y is not bound in H t does 
it add the variable to the scan-set S. This ensures that a variable moves at most once from the 
from-heap to the scan-set. 

Formulating the free-variable tracing algorithm as a rewriting system is easy. It requires only 
one rule that relates triples of from-sets, scan-sets, and to-sets: 

{H f W{x = h},S\&{x},H t ) =J> (H f ,Su(FV(h)\{Dom{H t )\±){x})),H t \+l{x = h}} 

Initially the free variables of the evaluation context and instruction expression, which correspond 
to the "roots" of a computation, are placed in S. Computing the free variables of the context 
represents the scanning of the "stack" of a conventional implementation while computing the free 
variables of the instruction expression corresponds to scanning the "registers." The initial tuple is 
re-written until we reach a state where no variable in the scan-set is bound in the from-heap. At this 
point, we have forwarded enough bindings to the to-heap. This leads to the following free- variable 
tracing algorithm rule: 

letrec H in e letrec H' in e 

(fva) if 

{H, FV{e),<D) =>* (H", 5, H') and Dom{H") n S = 0 
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Clearly, the algorithm always terminates since the size of the from-heap strictly decreases with each 
step. Furthermore, this new rewriting rule is a subset of the rule fv, which implies the correctness 
of the algorithm. 

Theorem 4.5 If P ^ P' , then P ^ P' . 

Proof: Let P = letrec H in e be a Age program. The first step is to prove the basic invariants of 
the garbage collection rewriting system. If (H, FV(e), 0) =>* (H/,S,H t ), then Hj l+J H t = H and 
PF(letrec H t in e) = S. Now let P' = letrec P 2 in e and suppose P ^ P'. Then, 

(Pi l+J H 2 , FV{e),%) =>* (P 2 , 5, Pi). 

and Dom(H 2 ) n 5 = 0. By the invariants, PF(letrec Pj in e) = 5, so Dom(H 2 ) n PV(P') = 0. 
Consequently, P h^-> P'. □ 

If we require that a collection algorithm produce a closed program, then fva is an optimal 
collection algorithm. That is, if P is a closed program and P P', then P' has the fewest 
bindings needed to keep the program closed without affecting evaluation. Assuming each step in 
the free-variable tracing algorithm takes time proportional to the size (in symbols) of the heap 
object forwarded to the to-heap, the time cost of the algorithm is proportional to the amount of 
data preserved, not the total amount of data in the original heap. 

4.3 Generational Garbage Collection 

The free- variable tracing algorithm examines all of the reachable bindings in the heap to determine 
that a set of bindings may be removed. By carefully partitioning the heap into smaller heaps, a 
garbage collector can scan less than the whole heap and still free significant amounts of memory. 
A generational partition of a program's heap is a sequence of sub-heaps ordered in such a way that 
"older" generations never have pointers to "younger" generations. 

Definition 4.6 (Generational Partition) A generational partition of a heap H is a sequence 
of heaps Hi, H 2 , ■ . ■ , P n such that H = H t i+l H 2 W ■ • ■ W H n and for all i such that 1 < i < n, 
FV(H{) (~1 Dom(Hi + i l+J P I+2 1+) • • • l+l H n ) = 0. The Hi are referred to as generations and Hi is said 
to be an older generation than Hj if i < j. 

Given a generational partition of a program's heap, a free-variable based garbage collector can 
eliminate a set of bindings in younger generations without looking at any older generations. 

Theorem 4.7 (Generational Collection) Let H x , . . . ,H{, . . .,H n be a generational partition of 
the heap of P = letrec H in e. Suppose H t = {H} l+l Hf), and Dom{H?) D PV(letrec H\ l±J H i+ i l+l 
■ ■. • l+» H n in e) = 0. Then P -^ letrec (P \ Hf) in e. 

Proof: We must show that Dom(Hf) D FF(letrec (P \ Hf) in e) = 0. Since H u ■ ■ -,H n is a 
generational partition of P, for all j, 1 < j < i, FV(Hj) n Pom(P J+1 l+l • • • 1+) H n ) = 0. Hence, 
FV(Hi ttl • • • l±J Hi-i) n Dom{Hf) = 0. Now, 

PI/(letrec P \ Hf in e) n Dom{Hf) 

= {FV{H \ Hf) U FV(e)) n Dom(Hf) 

= (FV{Hi 1+) • • • l±J Hi-i) U FV{H} l±) • • • l±l H n ) U FV(e)) n Dom{Hf) 

= (FV(Hi W • • • l+J P 8 _i) n Dom(Hf)) U ((PV(P/ l+l • • • l+J P„) U PF(e)) n Dom(Hf)) 

= 0 U ((PF(P/ l+l • - • l+) P n ) U F7(e)) n Dom(Hf)) 

= P7(letrec P/ l±J • • • l+J P n in e) n Dom(Hf) 

= 0 
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□ 

Generational collection is important for three practical reasons: First, evaluation of closed, pure 
Age programs makes it easy to maintain generational partitions. 

Theorem 4.8 (Generational Preservation for R) Let P — letrec H in e be a closed program. 
If Hi,..., H n is a generational partition of H and P letrec H 1+1 H' in e, then Hi, . . . , H n , H' 
is a generational partition of H l±l H' . 

Proof: The proj rule does not modify the heap, so it clearly preserves the generational partition. 
If the alloc rule is applied, then e = E[K] for some E and h and H' = {x = h}. Since P is closed, 
x cannot occur free in H. Thus for all i, 1 < i < n, FV(Hi)nDom(H i+ i \&H i+2 W- • • l±J if n l±l #') = 0. 
If the app rule is applied, then e = Elx y] for some E, x, and y such that x is bound to Xz.e' and 
y is bound to some h in H. Thus, H 1 = {z — h} and since P is closed, z cannot occur free in H. 
Thus for all i, 1 < i < n, FV(Hi) n Dom(H i+ i l+l H i+2 1+) ■ • • l±J jfiT n W H') — 0. □ 

The second reason generational collection is important is that, given a generational partition, 
we can directly use the free- variable tracing algorithm to generate a collection of a program. The 
following rule invokes the free- variable algorithm on the program letrec H2 in e where Hi,H 2 is 
a generational partition of the original program's heap. The resulting heap is glued onto Hi to 
produce a collection of the program. 

letrec H in e ^ letrec Hi l±) H' 2 in e 
(gen) if letrec ff 2 in e letrec #3 in e 

and fli, #2 form a generational partition of H 

The rule's soundness follows directly from the Generational Collection theorem, together with the 
soundness of the free-variable tracing algorithm. 

Theorem 4.9 If P ^ P', then P ^ P' . 

The third reason generational collection is important is that empirical evidence shows that 
"objects tend to die young" [35]. That is, recently allocated bindings are more likely to become 
garbage in a small number of evaluation steps. Thus, if we place recently allocated bindings in 
younger generations we can concentrate our collection efforts on these generations, ignoring older 
generations, and still eliminate most of the garbage. 

4.4 Assignment and Generational Garbage Collection 

The addition of certain language features or implementation techniques such as assignment or 
lazy evaluation tends to break the Generational Preservation theorem. For instance, consider the 
addition of an assignment operator (:=) to Age: 

(expressions) e € Exp ::= • • • | e x := e 2 
(contexts) E € Ctxt ::= • ■ • j E := e \ x := E 
(instructions) / € Instr ::= ■■•\x:—y 

(set) letrec H l±J {x = h, y = h'} in E\x:=y\ ^> letrec H W {x = h',y=h'} in Elxl 

resulting in a language Age-set. Evaluation of Age-set programs under R + set does not satisfy the 
Generational Preservation theorem. To see this, consider the program: 

letrec {x 0 = 0, Xi = 1, x 2 = (x 0 , x 0 ), x 3 = (z 2 , x 2 )} in 7r x (7^ (xi := x 3 )) 
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where the heap can be generationally partitioned into Hi — {xq = 0, X\ = 1} and H 2 = {X2 = 
(xq, xo), x 3 = (x 2 , x 2 }}. Performing the assignment results in the program: 

letrec {x 0 = 0, Xi = (x 2 ,x 2 ),x 2 = {x 0l x 0 ), x 3 = (x 2 , x 2 )} in n l (7^ xi) 

but taking Hi = {x 0 = 0, Xi = (0:2, 2:2}} and H 2 = {x 2 = (x 0 , x 0 ), x 3 = (x 2 , x 2 )} no longer yields a 
generational partition. If we attempted to do a generational collection on H 2 using this partition, 
then both x 2 and x$ would be collected since there is no reference to them in the expression part 
of the program. This would result in the program: 

letrec {x Q = 0, Xi = (x 2 ,x 2 )} in 7Ti (xi Xi) 

which after one proj step would be stuck: 

letrec {xq — 0, x x = (x 2 , x 2 )} in tti x 2 

because x 2 is unbound. Yet, if no collection was performed, the computation would not get stuck 
and would return an answer of 0. 

It is possible to maintain a generational partition for Age-set during evaluation. We simply keep 
track of all "older" bindings that are updated and move them from the older generation to a younger 
generation. The following theorem formalizes this idea for two generations. The generalization to 
multiple generations is straightforward. 

Theorem 4.10 (Generational Preservation for i? + set) If P = letrec Hi\$H 2 in e is a closed 
Xgc-set program and P h±f^*p' where P' = letrec Hi l±l H' 2 l+J Hz in e' and Dom{H 2 ) = Dom{H' 2 ), 
then Hi, (H 2 W H3) is a generational partition of P' . 

The theorem states that if we evaluate P for some number of steps resulting in P', where Hi^H 2 
is conceptually the older generation, H 2 is the subset of the older generation that is modified via 
set to become H 2 , and H3 is the younger generation of newly allocated bindings, then Hi and 
(H 2 l+J H3) form a generational partition. There are a variety of techniques for determining which 
values in the older generation must be moved to the younger generation. We refer the interested 
reader to Hosking et al. [19]. 

5 Garbage Collection via Type Recovery 

The delimiters and other tokens of the abstract syntax mark or "tag" heap values with enough 
information that we can distinguish pairs from functions, pointers from integers, etc.. This allows 
us to navigate through the memory unambiguously, but placing tags on heap values and stripping 
them off to perform a computation can impose a heavy overhead on the running time and space 
requirements of programs [33]. An alternative to tagging is the use of types to determine the shape 
of an object. If types are determined at compile time and evaluation maintains enough information 
that the types of reachable objects can always be recovered, then there is no need to tag values. 

A number of researchers have made attempts to explore this alternative [7, 8, 3, 15, 26, 34, 2], 
but none of them presented concise characterizations of the underlying techniques with correctness 
proofs. In this section, we present the basic idea behind type-recovery based garbage collection. In 
the following subsection, we give a simple overview of the techniques used to record and reconstruct 
the types of objects in collectors such as Britton's [8] and Tolmach's [34]. We then introduce 
Age-mono, an explicitly typed, monomorphic variant of Age. We show how to adapt the free- 
variable tracing algorithm to recover types of objects in the heap and to use these types in the 
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traversal of heap objects instead of abstract syntax. The proof of correctness for this garbage 
collection algorithm is given by extending the proof of soundness of the type system for Age-mono. 
The last subsection extends the work to an explicitly typed, polymorphic language. 

5.1 An Overview of Type Recovery 

The type of a data structure such as a tuple or integer determines the types of the components 
of the data structure (if any), but the type of a function does not contain the types of the free 
variables of the function. If the compiler associates the types of all free variables with the code 
of a function, it becomes possible to recover the types of all reachable objects in the heap, given 
the set of reachable objects from the current evaluation context and instruction. This provides a 
potential for space savings compared to tagging all values, because type information, like code, can 
be shared among different applications of the same function. 

For example, in a low-level model that uses closures (an explicit environment mapping variables 
to values paired with an expression) instead of substitution, the expression part or "code" of the 
closure can be shared. The type information needed to find and preserve the free variables of the 
closure is simply a type environment that lists the types of each value in the closure's environment. 
This information can also be shared because the types of the free variables are the same for each 
value environment paired with a given expression. 

As noted earlier, determining the free variables of the evaluation context and instruction cor- 
responds to scanning a stack and registers in a conventional implementation. Thus, the compiler 
must provide a type map for each stack frame. Like the type environment associated with a closure, 
this information can be recorded at compile time and a pointer to the information can be pushed 
on the stack as a new stack frame is allocated. The type information is "collected" as stack frames 
are popped. 

In our high-level framework, we use a global heap instead of closures, rely upon a-conversion to 
generate unique names, and use evaluation contexts instead of an explicit stack to model evaluation. 
Consequently, the set of free variables changes as expressions are evaluated and a-conversion is 
performed. Thus, the type maps associated with functions and evaluation contexts must also 
change. Our approach is to represent these type maps implicitly by initially tagging each expression 
with its type and by allowing the substitution that occurs as part of a-conversion to replace a 
variable with a variable, but leave the type tag intact. The type map information associated with a 
closure or evaluation context can then be extracted by a simple function similar to the FV function 
of Figure 2. As data structures are allocated, the type information is stripped, leaving the heap 
essentially tag free and ensuring that we do not use the abstract syntax to navigate the heap. 
However, the type information remains intact within functions, corresponding to the explicit type 
maps needed for closures. 

5.2 Age-mono 

Age-mono is an explicitly typed, monomorphic variant of Age. The set of types (r) of Age-mono 
contains the conventional basic types and constructed types for typing a functional programming 
language like Age. The expressions of Age-mono are the same as for Age, except that each raw 
expression (u) is paired with some type information (see Figure 5). Heap values are not paired 
with their type, reflecting the fact that the memory is "almost tag free." 

The evaluation contexts (E) consist of a raw context (U) and a type (r). Raw contexts are 
filled with instructions (7) which are a subset of the raw expressions (u) . The evaluation rules for 
Age-mono, named RM, are variants of the rules for Age that largely ignore the type information 
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Types: 

(types) r G Type ::— int | T\ x t 2 \ T\ — > r 2 

Programs: 



(expressions) 


e 


G 


TExp 


:= (u : r) 




u 


e 


UExp 


:= x | i | (ei,e2) | Jr. e | Aa;:r.e 


(heap values) 


h 


G 


Hval 


:= i | (x\, X2) | Ax:r.e 


(heaps) 


H 


e 


Heap 


:= {xi = hi, . . . ,x n = h n ] 


(programs) 


P 


G 


Prog 


~ letrec H in e 


(answers) 


A 


G 


Ans 


:= letrec i/ in (2; : r) 



Evaluation Contexts and Instructions: 

(contexts) E G TCtxt ::= (J7 : r) 

U G tfCtorf ::= C ] | (E,e) | <(x:r),£) | jr.- £ | E e \ (x:r) E 
(instructions) I G Instr ::= i \ ((xi:ti), (xo'-t^)) \ Xx:r.e | 7Ti (x:r) | 7r 2 (x:t) [ (x:Ti) (y:r 2 ) 



Figure 5: The Syntax of Age-mono 



on the sub-expressions of the program's body. 

(alloc-int) letrec H in Eli] ^4 letrec H l±) {a; = z} in i?[x] 

(alloc-pair) letrec in £'[(x 1 :r 1 , X2:t 2 )] letrec H l+J {x = (xi,x 2 )} in £[i] 

(alloc-fn) letrec in ElXy.r.e] y— ^ letrec # l+l {x = Aj/:r.e} in £T.x] 

(proj) letrec i7 in Eln, (x:r)] 1— letrec H in E'Cx,-] (H(x) — (xi,x 2 ),i = 1,2) 

(app) letrec # in £T(x:ti) (y:r 2 )] letrec # l±l {z = £T(y)} in Elu] (H{x) = Xz:r' .{u:r")) 

Allocation of integers, tuples and functions strips the type tag off of the heap value before placing 
it in the heap. Allocation of tuples also removes the tags from the components of the tuple. The 
removal of type information corresponds to the passage of a value from code to data and does not 
necessarily reflect a runtime cost. Projection and application are essentially the same as for Age. 
Note that substitution of a result expression for an instruction occurs "in place," and hence the 
type of the instruction is ascribed to the expression. 

The notion of a stuck state is adapted in accordance with the type structure of the language. 

Definition 5.1 (Age- mono Stuck Programs) A program is stuck if it is of one of the following 
forms: 

• letrec H in Ebr{ (x:r)] (x £ Dom(H) or H(x) ^ (21,2:2)); 

• letrec H in E[(x:ti) (y:r 2 )] (x G" Dom(H) or H{x) ^ Xz:r.e or y £ Dom(H)). 



5.3 Static Semantics 

The static semantics of Age-mono consists of four judgements about program components. The first 
judgement, F > e, is a binary relation between a type assignment T, mapping a finite set of variables 
to types, and a typed expression e. Figure 6 (Expressions) contains the fairly conventional inference 
rules for expressions that generate the static semantics. Typing heaps and complete programs 
requires three additional judgements. The first is T 0 h : r which asserts that the heap value h 
has type r under the assumptions in T. The second is F t> H : F', which asserts that the variables 
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Expressions: 



( Var ) ru TIZ\ ^<„.^ ( int ) 



r l±J {x:t} > (x : t) T t> [i : int) 



r>(m:ri) r>(w 2 JT2) , [ > (tt : n x r 2 ) 

(palp) r>({( U2 :r 1 ) ! ( U2 :r 2 )):r 1 xr 2 ) (pr ° j) r > (« : n x r 2 ) : 

ri±) {x-.tx} > (u : r 2 ) T > (mi : n r 2 ) T > (u 2 :ti) 

^ T > (Ai:ri.(w:r 2 ) : r x r 2 ) T > ((u : : n -> r 2 ) (u 2 :n) : r 2 ) 

Heaps and Programs: 

(h-int) 



r > i : int 



T>(x 1 :t 1 ) T>(x 2 :t 2 ) T > ( Az:n.e : n -» ^2) 
(h-pair) — - — (h-ln) — — - 

v ; r>(ii,i 2 ):nxT 2 r > Xx:n.e : t x r 2 ) 

Va;€l>om(r').ri±)r'> J fir(a:) :r'(x) , . 0>ff:r r> e 

(heap) r.F:r (Pr ° g) > letrec g in e 

Figure 6: The Static Semantics of Age-mono 



given type r in V are bound to heap values in H of the appropriate type, under the assumptions 
in T. The judgement's definition via the heap rule, similar to Harper's store typing [18] and the 
typing for Wright k Felleisen's Reference ML [37], requires "guessing" the types of the values in 
the heap and verifying these guesses simultaneously due to potential cycles. The third judgement, 
> P, asserts the well-typing of a complete program. Figure 6 (Heaps and Programs) contains the 
necessary inference rules for all of these judgements. 

Definition 5.2 (Well Formed Age- mono Programs) A Xgc-mono program P is well formed if 
[> P is derivable. 

The calculus Age-mono is type sound in that evaluation of well formed programs cannot get 
stuck [37]. Our proof of soundness uses a subject-reduction based argument in the style of Wright 
and Felleisen [37]. 

Theorem 5.3 (Type Soundness) If > P then either P is an answer or else there exists some 
P' such that P^P' and > P' . 

Proof (sketch): The first two lemmas (below) are needed to prove properties about the allocation 
rule. The Weakening lemma allows us to throw extra information into a type assignment and still 
type check an expression; the Allocation lemma allows us to replace an expression with a variable, 
as long as that location is placed in the type assignment with the proper type. 

Lemma 5.4 (Weakening) 7/T t> e and x <£ FV{e), then V l±l {x:t} o e. 

Lemma 5.5 (Allocation) If T > (u : r) and F > Elu] and x £ Dom{Y), then Y l±J {x:r} t> 
EI(x:t)1. 

The Unique Decomposition lemma allows us to deconstruct a program into a unique evaluation 
context and instruction expression. 
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Lemma 5.6 (Unique Decomposition) If P = letrec H in e has no free variables, then either P 
is stuck, P is an answer, or else there exists unique E and I such that e — E [/] . 

Lemma 5.7 (Stuck Programs Untypeable) If P is a stuck program, then ft P. 

Finally, the key lemma is Type Preservation, which asserts that if a program is well typed and it 
takes a step, then the resulting program is also well typed. 

Lemma 5.8 (Type Preservation) If > P and P i — > P' , then > P' . 

Proof (sketch): Follows from the Unique Decomposition, Weakening, and Allocation lemmas, 
together with an examination of each of the rules of RM. □ 

Given these lemmas, the proof of type soundness is straightforward. Suppose > P and P does not 
diverge. Then there exists some P' such that P ^^.*p' and P' is canonical with respect to RM. 
Further suppose that P' — letrec H in e for some H and e. If e = {x:r), then by Type Preservation, 
> letrec H in x:r and the theorem is satisfied. Otherwise, by the Unique Decomposition lemma, 
P' must be stuck and thus t> P' is not derivable, contradicting type preservation. □ 



5.4 Using Types in Garbage Collection 

Specifying a valid garbage collection rule that exploits types is now straightforward. The key 
property that the rule must preserve is type preservation. That is, if P P', and P is well typed, 
then P' must be well typed. One way to achieve this goal is to make it into a side-condition of the 
GC rule: 

(mono) letrec Hi l±l H 2 in e letrec Hi in e if > letrec Hi in e 

Theorem 5.9 For all well formed Xgc-mono programs P, (P, RM) ~ (F, RM + mono). 

Proof: Since mono preserves types, the type soundness proof trivially extends to the dynamic 
semantics with mono. This implies that, since P is well formed, P cannot get stuck under either 
system. Since bindings are only dropped and not updated by mono, the results of evaluating under 
either system must be the same. □ 

Like the free-variable rule of Age, mono needs to be refined before it can serve as the basis 
for an implementation. The refinement is similar to the one from fv to fva but uses the types 
embedded in programs to traverse heap values. The basis for the collection algorithm is a function 
that determines a minimal, with respect to set-inclusion, type assignment T for any expression e 
such that T t> e: 

MTA(x:t) = {x:t} 
MTA{v.t) = 0 
MTA{(ei, e 2 ) : n x r 2 ) = MTA{ei) U MTA{e 2 ) 

MTA(TTi e:r) = MTA(e) 
MTA{Xx:n.e : r x r 2 ) = MTA{e) \ {x:ti} 

MTA(ei e 2 : r) = MTA(ei) U MTA(e 2 ) 

Lemma 5.10 IfT>e, then MTA(e) > e and MTA(e) C T. 
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If P = letrec H in e and P is closed, then MTA(e) determines the types of the locations in the 
heap H that are immediately reachable from the expression e. A garbage collector can use this 
type information together with the following Tag function to traverse the reachable heap values 
based on their types instead of their abstract syntax. 

Tag[mt](i) = (e:int) 

Tag[n x t 2 ]((x u x 2 )) = ((fan), {x 2 :t 2 )) : n x r 2 ) 

Tagln -4 T 2 ](\x:T 1 .e) = ((Axrrj.e) : n r 2 ) 

Tag is a curried function that takes a type and then takes a heap value of that type, and annotates 
that heap value with enough information to turn it back into an expression. It is important to note 
that Tag pattern-matches and operates according to the type argument given and not the abstract 
syntax of the heap value given. MTA can be used on the resulting expression to find the minimal 
type assignment for the heap value. This provides us with the free locations and their types for 
the heap value. The following lemma summarizes the relationship between Tag and MTA: 

Lemma 5.11 // 0 > : T 1+) {a;:r} then there exists an h such that 

1. {x = h}CH 

2. MTA(Tag[T]{h)) > h : r 

3. MTA(Tag[r]{h)) C T l+l {x:r} 

Equipped with MTA, we can now redefine the free- variable tracing algorithm so that it uses Tag 
to traverse heap values. The algorithm is specified in the same manner as fva, i.e., as a rewriting 
system among tuples of the form (Hj, T s , H u T t ) where H f is the from-heap, V s is a type assignment 
corresponding to the scan-set, H t is the to-heap, and T t is a type assignment for the to-heap: 

(Hj l±J {x = h}, T s W {x:r} } H u T t ) =3? <#/, T' s , H t V{x = h}, T' t ) 

where T' s = {T s U MTA(Tag[r](h)) \ T' t 
Y' t = T t \±i{x:r} 

The algorithm is initialized by taking the program heap H as the initial from-heap and the minimal 
type assignment of the program expression as T s . At each step in the algorithm, T s describes the 
types of all locations that are immediately reachable from e or H t , but have not yet been forwarded 
to H t . r t describes the types of all locations that have been forwarded to H t . 2 

When a variable x is found in F s with type r and x is bound in Hj to the heap value h, the 
collector forwards the binding x = h to H t and adds x:r to r f . It then uses Tag[r] to traverse 
h. placing the necessary type information on the components so that MTA can determine the 
heap value's minimal type assignment. This step provides the locations (and their types) that are 
immediately reachable from h. Finally, the collector adds each of these locations to T s unless they 
have already been forwarded to H t . 

Using this algorithm instead of an a priori partitioning of the heaps, mono-a becomes a high- 
level specification of a collector whose traversal of heap values uses types instead of tag information: 

letrec H in e ~* letrec H' in e 
(mono-a) if 

{H,MTA{e),M)2& m (H",Q,H',T) 

The following definition gives the primary invariants of the algorithm: 

2 The garbage collection rewriting system only maintains T t to simplify the presentation and proof; an implemen- 
tation will not have to construct IV 
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Definition 5.12 (mono-a Invariants) (Hf,T s , H t ,T t ) has the mono-a invariants with respect 
to a program letrec H in e and type assignment T 0 iff: 

1- H = Hj 1+1 H t (each binding is either in the from-heap or to-heap.) 

2. r s 1+1 T t t> e (every binding needed for e is in the scan-set or to-heap.) 

3. Dom{T s ) C Dom(Hj) (the scan-set corresponds to bindings in the from-heap.) 
4- T s t> H t : T t (the scan-set holds all free variables in the to-heap.) 

5. T s 1+1 T t C To (the scan-set and to-heap agree with To.) 

Lemma 5.13 7/0 > H : Tq. T has the mono-a invariant properties with respect to P and To, and 
T T', then V has the mono-a invariant properties with respect to P and F 0 . 

Proof: Assume P = letrec H in e and T = (H } 1+) {x = h}, T s l+J {x:t}, H u T t ) and this rewrites 
to V = (H f , T' s , H t H){x = h}, T' t ) where r{ = T t a {x:t} and T' s = {T s U MTA{Tag[r]{h))) \ T' t . We 
must show: 

1. H = Hf l+J H t l+J {x = /i}: Follows directly from the first invariant for T. 

2. tt) t> e: Follows directly from the second invariant for T together with weakening. 

3. Dom{T' s ) C Dom(Hj): This follows from the third invariant if (Mr J 4(Ta P [r](/ i )) \ T' t ) C 
Dom(Hj). This in turn follows from > if : r 0 , {a;:r} C F 0 , invariant 5 for T, and Lemma 
5.11. 

4. r' s > H t tt) {2; = /i} : r' t : This follows from the fourth invariant of T and the heap rule of the 
static semantics if we can show (r s U (MTA(Tag[r](h)) \T' t )) \&T' t > h : r. This in turn follows 
from {x:t} C T 0 and Lemma 5.11. 

5. r; l+irj C r 0 : This follows from the fifth invariant of T if MTA{Tag[r]{h)) C r 0 . This in turn 
follows from {x:t} C r 0 and Lemma 5.11. 

□ 

Lemma 5.14 If 0 i> H : r 0 , r 0 > e, and (H, MTA(e), 0, 0) =3? *T ; i/zen T has the mono-a 
invariant properties with respect to H and To . 

Proof: By induction on the length of the rewriting sequence (H, MTA(e),$, 0) The base 

case directly follows from Lemma 5.10. The inductive step directly follows from Lemma 5.13. 
□ 

The correctness of the algorithmic type-based garbage collection rule can now easily be verified. 
Theorem 5.15 If P is a well formed program and P P' . then P P'. 

Proof: We must verify that the mono-a garbage collection rule preserves typability in order for 
mono to apply. That is, we must show that if P is a well formed program, and P T^ a P', then 
> P'. 

Let P = letrec H in e and suppose > P. Then, for some r 0 , 0 >H : r 0 and r 0 > e. If 

(H, MTA{e)X 0) ^*(H", 0, H', T t ), 

then by Lemma 5.14 taking T s = 0, we know that 0 > H ' : F t (invariant 4) and T t > e (invariant 2) 
. Thus, by the prog rule of the static semantics, > letrec H' in e. □ 

Furthermore, the algorithm never gets stuck, so it always applies: 
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Theorem 5.16 If P is a well formed program, then there exists a program P' such that P p\ 

Proof: If the algorithm takes a step, the size of the from-heap strictly decreases, so we know the 
collection either terminates or gets stuck. Here we show that the collection cannot get stuck, so 
it must terminate. Suppose P = letrec H in e and (H, MTA(e), 0, 0) ^*(Hf, T s l+J {x:t}, H u T t )- 
By Lemma 5.14, we know that x G Dom(Hf), since the domain of the scan type-assignment is 
a subset of the from-heap. Consequently, there exists an h such that Hf = Hf 1+1 {x = h} and 
(H f , T s t+J {z:r}, H t , T t ) ^*{H' } , T' s , H t H){x = h}, T' t ) with appropriate T> and T' s . □ 

The mono-a rule does not perform type inference but rather type recovery. That is, the types 
of values in the heap are recovered without unification. This was accomplished by making the types 
of subexpressions explicit in the program. We have traded tags on data structures such as pairs, 
with tags on sub-expressions of functions. However, the tags can be shared among functions that 
use the same code, differing only in their environment. Furthermore, some space savings can be 
realized for parametric data structures such as lists. For example, if we have x : (int X int) list, as a 
subexpression, x may be bound to a large list, but each of the components of the list have the same 
type and consequently share the same "type tag" in an expression. A true comparison between 
tag-oriented and type-oriented garbage collection will have to be based on implementations. 

5.5 Explicit Polymorphism 

Extending type-recovery based garbage collection to an explicitly polymorphic language where types 
are passed to polymorphic routines at run-time is straightforward. The language of the extended 
calculus, Agc-poly, is an adaptation of the Girard-Reynolds polymorphic A-calculus. The extended 
language of types contains type variables and quantified types: 

(type variables) t € Tvar 

(types) t £ Type ::= ■ • • | t | Vi.r 

The marker V binds a type variable t in a type expression r. Types that only differ in their choice 
of bound type variables are considered to be equivalent. FTV(r) denotes the free type variables of 
type r. 

The syntactic categories for Agc-poly are the same as for Age-mono, except for the following 
additions: 

u € untyped expr ::= • • • | At.e \ e [t] 
h € heap values ::= • • • | At.e 

The A-expression, or type abstraction, is a construct for introducing and scoping polymorphism; 
type application (e [r]) utilizes type abstractions at concrete types. 

The set of evaluation contexts for Agc-poly is the same as for Age-mono, modulo the extended 
syntax. In particular, there is no evaluation context within A-abstractions, because type abstracted 
expressions are allocated heap values. The one-step rewriting rules are the same as for Age-mono 
with the addition of two new rules, one for allocating A-abstractions and one for governing the 
application of A-abstractions: 

(alloc-t) letrec H in ElM.e] ^ letrec H 1+1 {x = At.e} in Elx] 

(type-app) letrec H in El{x:r) [r']] letrec H in EL{r'/t}ul (H(x) = At.(u : r")) 

As with other heap values, type abstracted expressions are allocated on the heap. Type application 
((x:r) [r']) replaces all free occurrences of the bound type variable in a A-expression with a type. 
A compiler for this language might represent A-expressions in the same way that it represents 
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A-abstractions and treat type application the same as term application. Since types are passed to 
type abstractions during evaluation, we say that this semantics is a type-passing interpretation. We 
refer to the extended set of rules as RP. 

The static semantics for Agc-poly defines the following judgements: 



1. 


A t> r 


(type r is well formed.) 


2. 


A»r 


(type assignment T is well formed.) 


3. 


A; r > e 


(expression e is well formed.) 


4. 


A;T>h :t 


(heap value h is well formed.) 


5. 


A;T>H : T' 


(heap H is well formed.) 


6. 


> P 


(program P is well formed.) 



where A is a set of type variables and T is a type assignment. The first judgement asserts that the 
type t is well formed with respect to A in that the free type variables of r occur in A. The second 
judgement asserts that T is well formed with respect to A, in that all of the types in the range 
of T are well formed with respect to A. The third judgement asserts that an expression is well 
formed. The judgement is defined in the same fashion as the expression-level typing judgement for 
Age-mono, except that T is required to be well formed with respect to A, and the resulting type 
must also be well formed with respect to A. The following rules are added for A-abstractions and 
type applications respectively: 

AfrT Aa{t};r>(«:r) A; T > (u : W.r') A > r 

A; T r> (At.(u:r) : Vi.r) ( * * ' A; f > ((« : W.r') [r] : {r/t}r') 

Similarly, a rule is added to typecheck a A-abstraction as a heap value: 

AoT Al±){i};r> (u:t) 
A;T > At.(u:r) : Vf.r 

The last two judgements assert that heaps and programs are well formed and are denned by the 
following inference rules: 

Vz € DomjT') .A;T\iir' c> H(x) :T'(x) 0;0>ff : r T t> e 

A;Tr> H : T' > letrec H in e 

A program is well formed if the heap and expression each type check with an empty A. Conse- 
quently, there can be no free type variables in a well formed program. 

As for Age-mono, evaluation preserves typing, so if a program is well formed and then takes 
some number of steps, the resulting program will have no free type variables. Furthermore, the 
type system for Agc-poly is sound: 

Theorem 5.17 (Type Soundness) If > P then either P is an answer or else there exists a P' 
such that P ^> P' and t> P' . 

The type-based free-variable algorithm can be adapted for Agc-poly as follows. First, MT'A and 
Tag are extended to work with the new language constructs: 

MTA(At.e) = MTA(e) 
MTA{e[r]) = MTA{e) 

Tag\\ft.r](At.e) = (At.e : Vt.r) 
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The garbage collection algorithm itself remains unchanged: 

{H f H){x = h},T s ^{x:r},H u r t ) (H f , T' $ , H t W {x = h}, T' t ) 

where T' s = {T S U MTA(Tag[r](h)) \T' t 
T' t = T t \H{xvr} 

To prove the algorithm does not get stuck, we must show that we never find a variable in T s that 
is assigned an unknown type (t). But this follows from the well-typedness of the original program. 
Consequently, it is straightforward to adapt the proofs of termination and correctness of mono-a 
to show that the rule poly-a is correct and always applies. 

6 Collecting Reachable Garbage Using Type Inference 

So far, we have only considered specifications and algorithms for collecting unreachable bindings. 
In this section, we show that by using type inference during the garbage collection process, some 
bindings that are reachable can still be safely collected. That is, type inference can be used to 
prove that an object is garbage even though it is reachable. 
As a simple example, consider the following Age program: 

letrec {xi = 1, x 2 = 2, x 3 = (x 2 , x 2 ), x 4 = (zi, 23}} in 7rj x 4 

Every binding in the heap is accessible from the program's expression (ni x 4 ), so the free- variable 
based collection rules can collect nothing. But clearly the program will never dereference x 3 nor 
x 2 . The inference-based collection scheme described in this section will allow us to conclude that 
replacing the binding x 3 = (x 2 ,x 2 ) with x 3 = 0 (or any other binding) will have no observable 
effect on evaluation. That is, the inference collection scheme shows that 

letrec {xi -l,x 2 - 2,x 3 = 0,x 4 = (xi,x 3 )} in tti x 4 

is Kleene equivalent to the original program. Now by applying the free- variable rule, we can 
conclude that the binding x 2 = 2 can be safely collected. 

In subsection 6.1, we introduce type inference for Age based on the presentation of Mitchell [25, 
Section 4.5]. In Section 6.2 we show that if type inference can assign a binding an unconstrained 
type and the rest of the program still has a valid typing, then the binding can effectively be collected. 
Our proof of this theorem is based on the method of logical relations, a standard proof technique 
from type theory. (See Mitchell [25, Section 3] for an overview of logical relations and various 
references.) 

6.1 Age and Type Inference 

We start by considering the original Age language as an implicitly typed, monomorphic language, 
where the types of the language are the same as for Age-mono except for the addition of type 
variables: 

(types) r € Type ::= t | int | n X r 2 | r a r 2 

By implicitly typed, we mean that the terms of the language are not decorated with types as in 
Age-mono. We add type variables to the set of types so that each well-formed expression has a 
principal or most general type (explained below). 

Type inference is the process of decorating Age programs with types so that the resulting 
program type checks under the Age-mono rules. (Refer to Figures 1 and 3 for the syntax and 
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dynamic semantics of Age and Figure 6 for the typing rules for Age-mono.) Alternatively, we may 
directly specify a set of typing rules for Age programs by taking the typing rules for Age-mono 
and erasing the type information from the terms, resulting in the inference system of Figure 7. 
These rules define judgements of the form Their, where T is a type assignment and e is a Age 
expression. We use "h" instead of ">" to keep from confusing these typing judgements with the 
Age-mono and Age-poly judgements. 



Expressions: 



( var ) F-"7—rr— (int) 



T l±J {x:r} h x : t v ' T h i : int 

T h ei : n r h e 2 : r 2 T I- e : r x x r 2 

(tuple) (pro j)^ 5. (i = 12 ) 

T W {a;::-!} he:r 2 F h e : : n r 2 r h e 2 : n 

Th Ax.e : n -> r 2 v w; r h e x e 2 : r 2 
Heaps and Programs: 



, Vx£Dom(r').ri±>T'\- H(x) :T'{x) , ,0 

( hea p) ThH-.r — — (prog) " 

Figure 7: Type Inference Rules for Age 



h H : T r h e : r 
h letrec H in e : r 



A given Age expression can have multiple typing derivations according to these rules and con- 
sequently multiple typings, but an expression's typings may be ordered so that there is a most 
general, or principal typing and every other typing is an instance of this principal typing and is 
thus derivable. The ordering is defined in terms of a substitution: 

Definition 6.1 (Substitution) A substitution S is a function from type variables to type expres- 
sions. 

We write Sr to denote the type obtained by replacing each type variable t in r with S(t) and 
we write ST to denote the type assignment {x:Sr \ x:r g T}. 

Definition 6.2 (Instance, Subsumes, Principal Typing) A typing T' h e : r' is an instance 
of the typing T h e : r if there exists a substitution S such that ST C T' and Sr = r' . In such a 
case, we say that (T,r) subsumes (T',r'). A typing T h e : r is principal if for all other typings 
T'heir', (T,r) subsumes (T', r'). 

A consequence of the definition of a principal typing is that any instance of the principal typing 
is derivable: 

Lemma 6.3 If (T, r) is a principal typing for e, then for all substitutions S , ST h e : Sr. 

It is straightforward to see that principal typings for Age expressions are unique (up to de- 
conversion of type variables) . Consequently, we can search for the principal typing of a Age program 
in order to determine if there is any typing derivation of a program. The critical component of a 
type inference algorithm is an algorithm for unifying types and type assignments: 
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Definition 6.4 (Unifier, Most General Unifier) A unifier of a set of type equations E is a 
substitution S such that for all T\ = r 2 € E, Sri is syntactically equal to Sr 2 . A unifier S of E is 
more general than S\ if there exists a unifier S 2 such that Si = S2 o S . 

Lemma 6.5 (Unification) [31] Let E be a set of type equations. There is an algorithm Unify(E) 
that computes a most general unifier of E if one exists and fails otherwise. 

If Ti and T 2 are type assignments, then the unification algorithm can be used to compute 
a most general substitution S such that STi U Sr 2 is well formed, if such a substitution exists. 
(Recall that a well formed type assignment maps term variables to at most one type.) We simply 
compute Unify(E) where E = {n = r 2 | x:r x G I\ and x:t 2 6 T 2 }. We write Unify{Y l , T 2 ) for the 
substitution obtained in this manner. 



PT{x) = ({*:«},<) (* fresh) 

PT(i) = <0, int) 
PT(( ei ,e 2 » = let(ri,7i) = PlXei) 

in let <r 2l r 2 ) = PT(e 2 ) 
in let 5 = Unify(Ti,T2) 
\n{ST 1 UST 2 ,Sr 1 x St 2 ) 
PT(7n e) = let <I\ t) = PT{e) 

in let S = Unify({x:T},{x-ii x< 2 }) <2 fresh) 

in (ST,SU) 
PT(Xx.e) = let {T,n x t 2 ) = PT{(x,e)) 

in (r \ {x:n},Ti -> r 2 ) 
PT(e x e 2 ) = let {r^n) = PT( ei ) in let (r 2l r 2 > = PT(e 2 ) 

in let S = Unify(T 1 \i){x:Ti},T 2 ^{x:T2^-t}) (a;, afresh) 
in (STi UST 2 ,St) 

PTA(e) = let {T,r) = PT(e) in T 
PTT(e) let (T,t) = PT(e) in r 

Figure 8: An Algorithm for Computing the Principal Typing of an Expression 



Given a Age expression e, the algorithm PT(e) in Figure 8 computes the most general type 
assignment and type for that expression. The algorithm is similar to the MTA algorithm of Sec- 
tion 5.4, but uses the Unify(-) routine to compute most general substitutions that allow two type 
assignments to be merged. The variable case returns a type assignment mapping the variable to a 
fresh type variable and returns that type variable as the type of the expression. The integer case 
returns an empty type assignment and the integer type. The tuple case computes the principal typ- 
ings of the components of the tuple, and then computes a unifying substitution of their respective 
type assignments. This step is necessary because the same variable could occur within two different 
components, and the types of these two occurrences must unify. The substitution is applied to each 
component's principal type assignment and the union of the resulting assignments is returned as 
the principal assignment of the tuple. The type is calculated by applying the substitution to each 
of the components' principal types and then forming a tuple type. The type of an abstraction 
Xx.e is calculated by finding the principal typing of the expression {x, e). The type of the resulting 
expression contains the domain type of the abstraction paired with the range type. The pairing is 
needed since x may not occur in e and thus may not occur in the resulting type assignment T. The 
principal type assignment of the abstraction is simply T minus the occurrence of x:t\. 
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The following lemma establishes the key properties of PT(e): 

Lemma 6.6 If PT(e) = (T, r), then Their. Furthermore, ifV he:r' for some V and r', then 
PT(e) exists and PT(e) subsumes (T',t'). 

The algorithm for determining the principal typing of an expression can be easily extended 
to find a principal typing for a heap and a program. The former returns a principal pair of 
type assignments (r, T H ) for H, such that T \- H :Th and the latter returns the principal type 
assignment and type for the program: 

PT{{x 1 = h 1 ,...,x n = h n }) = let (T lt n) = PT^) ••• <r Bl T n > = PT(e n ) 

in let T = {xxin,- ■■,x n :T n } 
in let S = Unify{T,T u ---,T n ) 

in <(sr 1 u---usr n )\5r,sr) 

Pr(letrec H in e) = let (T,T H ) = PT{H) 

in let (r e ,r) = PT(e) 

in let 5 = Unify{Y l+l r#, T e ) 

in (sru{sr e \sr H ),Sr) 

The following lemma shows that PT(P) computes the principal type for a program and that any 
typing for the program is an instance of the principal typing. 

Lemma 6.7 If PT(P) = (T,r), then T h P : r. Furthermore, if V h P : r' , then (T,t) subsumes 

<rv>. 

The algorithm is complete in that if there exists some typing for the program, then the algorithm 
will not fail to find the principal typing of the program. 

Lemma 6.8 IfV h P : r, then PT{P) exists. 

Finally, soundness for the static semantics follows by showing that principal typings of a program 
are preserved by evaluation. 

Theorem 6.9 (Type Soundness) // h P then either P is an answer or else there exists a P' 
such that PA P' and h P' . 

6.2 The Inference GC Specification 

We will show that if we can find a typing for a program that assigns a location bound in the 
heap a type variable, then that location may be bound to any value without affecting evaluation. 
Consequently, any pointers contained in the location's binding do not need to be scanned and 
traced during garbage collection. The intuition behind the theorem is that a location's type is 
unconstrained only if the location is not applied nor projected nor used in some manner that would 
constrain the type. Consequently, we can replace the binding in the heap with any binding we 
choose. In particular, if the location is bound to a large heap value, we can bind the location to an 
integer or dummy piece of code without affecting evaluation. This replacement allows us to collect 
any bindings that used to be reachable through this binding without knowing anything about the 
shape of the original heap value. 

The proof of the theorem relies upon the semantic interpretation of types as logical relations. In 
our case, the relations are a type-indexed family of binary relations relating programs to programs, 
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answers to answers, and heaps to heaps. The relations are contrived so that, if two programs 
are related, then they are Kleene equivalent, so one program converges to an answer iff the other 
converges to a related answer and related answers at base type (int) yield equal values. Roughly 
speaking, the relations are logically extended to relate answers of higher type (— >■) if, whenever 
such answers are applied to appropriately related answers, the resulting computations yield related 
results. 

The relations are defined in Figure 9 by induction on types. The definitions are parameterized 
by an arbitrary relational interpretation of type variables, G. If £ is a type variable, then Q(£) 
determines some fixed, but arbitrary relation between answer programs. This is consistent with 
the idea that well- typed programs have an implicit "V" quantifier for the type variables in a program. 
The parameterization of the interpretation of type variables makes it straightforward to extend the 
definition of the relations to account for predicative polymorphism. 



Computations: 



0 (= P 1 ~ P 2 : T iff h Pi : t and h P 2 : r and 

Pi -U-jz Ai implies P 2 ii-R A 2 and 0 (= Ai sa A 2 : r 
and 

P 2 ii-R A 2 implies Pi JJ.^ A\ and 0 |= A 1 ss A 2 : r 



Answers: 



<d\=A l KA 2 :t iff (A u A 2 )ee(t) 

0 (= letrec Hi in x x « letrec H 2 in x 2 : int iff #1(2:1) = H 2 (x 2 ) = i 

0 (= letrec Hi in xi « letrec H 2 in x 2 : t x x t 2 iff 0 (= letrec Hi in iti xi ~ letrec H 2 in rci x 2 : n 

and 

0 |= letrec Hi in 7r 2 xi ~ letrec H 2 in 7r 2 x 2 : r 2 

0 (= letrec Hi in xi « letrec H 2 in x 2 : n -» r 2 iff s iH' l , H' 2 , yi , y 2 . 

0 f= letrec Hi t+l H[ in yi ss letrec H 2 1+) H' 2 in y 2 : t x 
implies 

0 (= letrec Hi W H[ in xi yi ~ 

letrec H 2 1+) in x 2 y 2 : r 2 

Heaps: 

0 (= #1 w i? 2 : T iff 0 h if! : T and 0 h ^2 : T and 

Va; G Dom(r).0 |= letrec ffi in x w letrec i? 2 in x : T{x) 

Figure 9: Relational Interpretation of Types 



Two well- typed programs Pi and P 2 are related at a type r, written G \= Pi ~ P 2 ■ r iff, 
whenever one of the programs terminates with an answer, then the other program terminates with 
a related answer at type r. 

Two answer programs Ai and A 2 are related at type r, written G \= A\ ^ A 2 : r, as follows: 
If t is a type variable t, then the answers are related iff they are in the relation Q(t). If r is an 
integer, then the answers are related iff, when we lookup the answer variables in their respective 
heaps, they are bound to the same integer value. If r is a pair, then the answers are related iff, 
whenever we project a component, the resulting programs are related. Finally, if r is an arrow type 
Ti -¥ 7-2, the answers are related iff, whenever we apply the answer variables to related arguments 
at type r x , we get related programs at type r 2 . Even though the relations between programs and 
answers are defined in terms of one another, the relations are well-founded because the size of the 
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type index always decreases when one relation refers to another. 

The definition of the relations ensures that related programs remain related even if more bindings 
are added to the programs' heaps. 

Lemma 6.10 If 0 f= letrec Hi in e x ~ letrec H 2 in e 2 : r and h letrec Hi l+J H{ in e : r and 
h letrec H 2 \H H' 2 \r\ e : t, then 0 |= letrec Hi&H[ in e ~ letrec ff 2 W # 2 in e : r - 

Since evaluation only adds new bindings and leaves existing bindings intact, it is clear that evalu- 
ation preserves the relations. If the language permitted assignment, then this property would not 
necessarily hold. 

For the statement of the following lemma, we need to extend the answer relation to heaps. Two 
heaps, Hi and H 2 , are related at a context T, written 0 \= Hi % H 2 : T, if they can be a-converted 
so that, for all variables x in F, the answers letrec Hi in x and letrec H 2 in x are related at T(x). 

The following lemma establishes our primary result: an expression is related to itself in the 
context of any two related heaps. 

Lemma 6.11 For all 0 : Tvar -> (Ans <-» Ans), if T h e : r and 0 \= Hi « H 2 : T, then 
0 |= letrec Hi in e ~ letrec H 2 in e : r. 

Proof: By induction on the derivation of T h e : r. 
var: (r ID {x:r} hi:r) By the definition of 9 |= Hi « # 2 : T. 

int: (r h i : int) letrec iJj in i 3-» letrec iJj ID {xj — i} in xj for j = 1, 2 and these two answers are related 
by the definition of 0 |= A\ ss A 2 : int. 

tuple: (T h (ei,e 2 ) : n x r 2 ) Suppose letrec in (e 1; e 2 ) does not diverge. Then by examination of the 
rewriting rules, it is clear that: 

letrec H x in (e 1) e 2 ) ^4* letrec #i ID H[ ID {x x = fti} in (xi,e 2 ) 3»* 
letrec #j. ID H[ ID {xi = h{] ID ID {x 2 = h 2 ] in (xi, x 2 ) 3> 
letrec #1 ID ID {x x = hi } ID H[' ID {x 2 = ft 2 } ID {x = (xi,x 2 )} in x 

Thus, letrec #i in e x 3** letrec i/'ilD/fJ ID{xi = fti} in x x . By the induction hypothesis applied to the 
first hypothesis of the pair rule, we conclude that letrec H 2 in t\ letrec H 2 ID.r7 2 lD{xi = ^'1} ' n 
and these two answers are related. Consequently, letrec H 2 in (ei,e 2 ) 3* * letrec £T 2 ID H 2 ID {xi = 
/i'j} in (xi,e 2 ). 

From the rewriting sequence above, we can conclude that letrec H\ ID H[ ID {xi = /i x } in e 2 3* 
"letrec Hi ID ID {x : = /ij.} ID i?" ID {x 2 = h 2 ] in x 2 . By hypothesis, Q \= H x ^ H 2 : T a,nd thus by 
Lemma 6.10 0 \= HiWH[\±) {xj = Z^} « H 2 ID # 2 ID {x 2 = h 2 } : T, so the induction hypothesis applies 
to the second hypothesis of the pair rule and we can conclude that letrec H 2 iSH 2 l ±){x[ = h[} in e 2 3* 
* letrec H 2 ID H' 2 ID {x' : = } 1+) if,' ID {x' 2 = /i' 2 } in x' 2 and these two answers are related. Consequently, 
letrec H 2 ID H' 2 ID {x' : = Zi'J in (x'j , e 2 ) -A * letrec H 2 \t)H 2 \±l {x[ = h[} ID ff 2 ' ID {x' 2 = ft' 2 } in (xi , x' 2 ). 
Since related answers remain related under heap extensions, 

0 (= letrec #i ID ID ID {xi = h 1 , x 2 = h 2 , x = (x x , x 2 , }) in x : - « 

letrec # 2 ID # 2 ID H' 2 ' ID {x' t = h[,x' 2 = h' 2 , x' = (x[,x' 2 , }) in x\ : n (i = 1, 2) 

Consequently, 

0 |= letrec #1 ID if { IDF" ID {xi = hi, x 2 = h 2 , x = {x 1; x 2 ,}) in 7T; x ~ 

letrec if 2 W # 2 ID H' 2 ' ID {xi = &i, x' 2 = h' 2 , x' = (x[ , x' 2> }) in tt; x' : n (i =1,2) 

Thus, 0 f= letrec i?i in (ei,e 2 ) ~ letrec if 2 in (ei,e 2 ) : n x r 2 . 
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proj: (r h 7T,- e : r,) Suppose letrec i?i in 7r,- e converges. Then, 

letrec i?i in 7r; e i-^letrec #1 l+J H[ in 7T,- 2; letrec #1 W in y 

for some x, and y. Thus, letrec Hi in e "letrec #! 1+1 i?( in a:. By the induction hypothesis, 
letrec H 2 in e i-^-*letrec H 2 W H 2 in a;' and 0 \= letrec #1 1+) in x w letrec # 2 W # 2 in z' : n x r 2 . 
By the definition of this relation, we know that 

0 |= letrec Hi 1+1 H[ in 7T; x ~ letrec # 2 W # 2 in ^ x> '■ r i x r 2 

for i = 1, 2. Thus, 0 (= letrec #1 in tt { e ~ letrec #2 in ^ e : n x r 2 . 

fn: (T h Xx.e : n r 2 ) letrec in Aar.e letrec Hi l+J {t/j = Xx.e} in y,- for i = 1, 2. We must show that 
these two answers are related. Suppose: 

0 |= letrec Hi l+J {yi = Xx.e} l±l H[ in 21 « letrec i?j l+J {y 2 = Xx.e} l+J ff 2 in z 2 : n 

for some H[, H' 2 , zi, and z 2 - We need to show that: 

© |= letrec Hi l+J {yi = Xx.e} l+J #J in yi zi ~ letrec Hi l+J {y 2 = Ax.e} l+J i? 2 in 2/2 *2 : r 2 

This follows if: 

0 (= letrec i7 : l+J {yi = Ax.e, 2; = hi} l+J in e ~ letrec JTi l+J {y 2 = Xx.e, x = fr 2 } l+J H' 2 in e : r 2 

where = (i? 2 - l±l {yi - Xx.e} l+J H<)(zi) for i = 1,2. By the induction hypothesis, it suffices to show 
that: 

0 |= Hi l+J {t/i = Xx.e,x = hi}\i)H[ « #1 l±J {y 2 = Aa:.e,ar = h 2 } W #2 : T l+J {ar : n} 
By the lemma's hypothesis, we know that 0 |= Hi « H 2 : T and thus by Lemma 6.10 
0 f= #1 l+J {j/i = Aar.e, a; = Ai} l+J #J « # 2 W {*/2 = Aar.e, x - h 2 } l+J ff 2 : I\ 

Since 

0 (= letrec Hi l+J {yi = Aar.e, a; = fti} l+J^ in z x w letrec Hi l+J {y 2 = Xx.e, x = h 2 } W # 2 in z 2 : n 
and Ai and /i 2 are bound to zi and z 2 in these respective heaps, it follows that: 

0 |= letrec Hi l+J {yi = Xx.e, x = hi} l+J H[ in x « letrec #i l+J {y 2 = Aar.e, a; = fc 2 } l+J # 2 in x : n 
Consequently, 

0 [= Hi l+J {yi = Aar.e, a: = /ii} l±l « #1 W {y 2 = Aar.e, ar = h 2 } l+J F 2 : V l+J {a; : rj 
holds and, working backwards we know that 

0 (= letrec Hi l+J {yi = Aar.e} l+J H[ in yi Zi ~ letrec #1 l+J {y 2 = Aar.e} l+J # 2 in y 2 z 2 : r 2 

and thus © |= letrec Hi l+J {yi = Aar.e} in yi w letrec # 2 l+J {y 2 = Aar.e} in y 2 : n ->• r 2 . 

app: (r H d e 2 : r 2 ) Suppose letrec #1 in e x e 2 does not diverge. Then by examination of the rewriting 
rules, it is clear that: 

letrec Hi in e x e 2 ^4* letrec Hi l+J H[ l+J {ari = hi} in a^ e 2 h^4* 
letrec Hi 1+1 ff( l+J {a?i = fti} l±J ffj' W {a; 2 = /i 2 } in x x x 2 

Thus, letrec ffi in e x >-^4*letrec Hi l+JF( l+l{a;i = fti} in ii. By the induction hypothesis applied to the 
first hypothesis of the app rule, we conclude that letrec H 2 in e x ^->* letrec H 2 l+J H 2 l+J {x\ = h[} in a;'! 
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and these two answers are related at r x -» r 2 . Consequently, letrec H 2 in t\ e 2 "letrec H 2 W H' 2 W 
{2^ = h[} in x[ e 2 . 

From the rewriting sequence above, we can conclude that letrec H\ l+J H[ 1+J {xi = /ii} in e2 

* letrec i7j (+J 1+) {xi = /ii} l±J E'{ l±l {x 2 = /» 2 } in a: 2 . By hypothesis, 6 |= H 1 « i? 2 : F and thus 
0 [= i?i W i7J l±) {a;i = /ii} w i7 2 1+1 H' 2 1+J {x 2 = h 2 } : F, so the induction hypothesis applies to the 
second hypothesis of the app rule and we can conclude that letrec H 2 W H' 2 l+J {x^ = h[} in e 2 

* letrec i^l+Ji^l+J^i = h[ }l+lF 2 l+l{a; 2 = /i' 2 } in x' 2 and these two answers are related at r x . Consequently, 
letrec J? 2 1±) H' 2 1+1 {a^ = /i'j} in e 2 A-*letrec # 2 l±l H' 2 l+J {x'j = />;} W # 2 ' 1±) {x' 2 = h' 2 } in x' 2 . 
Since 

6 |= letrec Hi 1+) H[ l+l {ari = /ii} l+J 1+1 {x 2 = /i 2 } in a; 2 « 

letrec H 2 l+l # 2 w = A 'i } W #2 W {4 = in x 2 '■ T u 

and 

0 |= letrec H 1 lSH[\S {x x = h x } l+l l±J {x 2 = /i 2 } in x x w 

letrec # 2 l+J # 2 l+l {ar'j = Zi'J (+) # 2 ' l+l {x' 2 = h' 2 } in ari : n -> r 2 

by the definition of « at arrow-types, it follows that 

0 1= letrec Hi l+l H[ l±l {xj = /ij 1+) l+l {x 2 = /i 2 } in n x 2 ~ 

letrec H 2 W £T 2 l+l = &i} l+l # 2 l+l {x' 2 = h' 2 } in x'j x' 2 : t 2 

Thus, 0 |= letrec H\ in d e 2 ~ letrec iifo in ei e 2 : r 2 . 



Our goal is to show that if we are given a context F, expression e, and heap H such that 
r h e : t and 0 h H : T, then letrec H' in e is Kleene equivalent to letrec H in e where if' is defined 
as follows: 

#'={x = I T(x) £ Tvar} l±) {x = 0 | 6 Tuar} 

This follows from Lemma 6.11 if we can show that, taking ©o(0 to be the everywhere-defined 
relation on answer programs, Go |= H « H' : T. This in turn follows if we can show that 
0o (= H fa : T (H is related to itself), since 0n(O relates every program and H'{x) differs from 
H(x) only when T(x) = i. 

Unfortunately, we cannot directly show that a well formed heap is related to itself! The problem 
is that if we attempt to argue by induction on the derivation of 0 h H : T, the uses of the heap 
rule require that we assume what we are trying to prove. The same problem is encountered when 
using logical relations to reason about conventional calculi with recursion or iteration operators. 

If we forbid cycles in the heap, then we can transform the derivation of the heap's well formedness 
into a derivation that only uses a let-style rule instead of the recursive letrec-style heap rule: 

M _ Ti Hi T 2 h h : r YihH:Y 2 

(let-heap) 



Yi h H l+l {x = h} : r 2 l±J {z:r} 

Consequently, if a heap is cycle free, we may show by induction on the derivation using the let-heap 
rule that the heap is related to itself. 

Lemma 6.12 IfYi h H : Y 2 , 0 (= H x « H 2 : Yi and H is cycle free, then 0 (= Hi l±J# ^ H 2 WH : 
Yi «F 2 . 

Proof: Since H is cycle free, there exists some derivation of Yi h i? : T using only the let- 
heap rule. The proof proceeds by induction on the height of this derivation. If H is empty, 
then the lemma holds trivially by the assumptions about Hi and H 2 . Suppose Fi h H : T 2 
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and ri t±J T 2 h h : r. By induction, 0 f= #1 1+) # w if 2 ttl : Ti tt) T 2 . By Lemma 6.11, 
0 (= letrec Fi ttl H in /i ~ letrec H 2 ttl if in h : r. Thus, 

0 (= letrec #! ttl i? ttl {z = /i} in x « letrec # 2 ttl # ttl {a; = /1} in z : r. 

Consequently, 0 [= #1 ttl il \t> {x = h} « # 2 « J5T ttl {x = h} : Ti ttl T 2 U {x:r}. □ 
Finally, we can state and prove the following Inference GC specification: 

Theorem 6.13 (Inference GC) Let T 1 = {xiiti, . . . , x n :t n } and Hi = {xi = hi, . . . , x n = h n } 
and H[ = {xi = 0, . . . , x n = 0}. If 

1. Ti ttl T 2 h e : r fr £ Tuarj, and 

2. I\ h if 2 : T 2j and 

3. 35.0 h Hi :STi, and 

4. H 2 is cycle free, 

then letrec Hi ttl H 2 in e ~ letrec i?( tt) i/ 2 in e. 

Proof: Taking @ 0 (t) to be the everywhere defined relation, 0 O |= Hi ^ H[ : Ti holds trivially. By 
Lemma 6.12, since H 2 is cycle free and r 1 \-H 2 :T 2 , we know that 0 O |= #itt)# 2 ^ H[HlH 2 : r^WlY 
Since Ti tt) T 2 h e : r, we know by Lemma 6.11 that 0 O f= letrec Hi ttl H 2 in e ~ letrec ff{ ttl 
F 2 in e : r. Thus, letrec Hi ttl # 2 in e letrec ff a in 2 iff letrec H[ ttl # 2 in e JJ- letrec in y 
and 0o (= letrec H a m x « letrec Hb'my : r. Suppose # a (z) = i (or #{>(y) = «')• Then r 
must be int since r £ Tuar. By the definition of « at int, H\>{y) = i (or H a (x) = i). Thus, 
letrec #1 tt) #2 in e ~ letrec H[ ttl # 2 in e. □ 

It should be possible to extend our argument to cyclic heaps if we can show that every cyclic heap 
is appropriately approximated by some finite "unwinding" of the heap. One approach is to use a 
fixed-point semantics for Age where types are interpreted as CPOs formed by suitably lifting answer 
programs. The meaning of an answer program letrec H in x would essentially be the least fixed- 
point of the chain of meanings of the approximations letrec H° in x, letrec H 1 in x, letrec H 2 in x,. . . 
where H i denotes the i ih unwinding of the heap H. See Gunter [17, Chapter 4.4] for an example 
of the treatment of unwinding and approximation in the context of a conventional A-calculus. 

7 Related Work 

The literature on garbage collection in sequential programming languages per se contains few papers 
that attempt to provide a compact characterization of algorithms or correctness proofs. Demers et 
al. [10] give a model of memory parameterized by an abstract notion of a "points-to" relation. As a 
result, they can characterize reachability-based algorithms including mark-sweep, copying, genera- 
tional, "conservative," and other sophisticated forms of garbage collection. However, their model is 
intentionally divorced from the programming language and cannot take advantage of any semantic 
properties of evaluation, such as type preservation. Consequently, their framework cannot easily 
model the type-based collectors of Sections 5 and 6. Nettles [27] provides a concrete specification 
of a copying garbage collection algorithm using the Larch specification language. Our specification 
of the free- variable tracing algorithm is essentially a high-level, one-line description of his specifica- 
tion. Like Demers et al., he uses a traditional definition of garbage based on unreachability in the 
memory graph and purposefully distances the specification from any language evaluation details. 
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Hudak gives a denotational model that tracks reference counts for a first-order language [20]. 
He presents an abstraction of the model and gives an algorithm for computing approximations of 
reference counts statically. Chirimar, Gunter, and Riecke give a framework for proving invariants 
regarding memory management for a language with a linear type system [9]. Their low-level 
semantics specifies explicit memory management based on reference counting. The goal of the 
work was to determine whether, using the linear type system, in-place update would be a sound 
optimization. Both Hudak and Chirimar et al. assume a fairly weak approximation of garbage 
(reference counts). 

Tolmach [34] built a type-recovery collector for a variant of SML that passes type information 
to polymorphic routines during execution, effectively implementing our Agc-poly language and the 
corresponding collector of Subsection 5.5. Aditya and Caro gave a type-recovery algorithm for an 
implementation of Id that uses a technique that appears to be equivalent to type passing [1] and 
Aditya, Flood, and Hicks extended this work to garbage collection for Id [2]. 

There have been a variety of papers regarding inference-based collection for monomorphic [7, 36, 
8] and polymorphic languages [3, 15, 16, 12]. Appel [3] argued informally that "tag-free" collection 
is possible for polymorphic languages such as SML by a combination of recording information 
statically and performing what amounts to type inference during the collection process, though the 
connections between inference and collection were not made clear. Baker [6] recognized that Milner- 
style type inference can be used to prove that reachable objects can be safely collected, but did not 
give a formal account of this result. Goldberg and Gloger [16] recognized that it is not possible 
to reconstruct the concrete types of all reachable values in an implementation of an ML-style 
language that does not pass types to polymorphic routines. They gave an informal argument based 
on traversal of stack frames to show that such values are semantically garbage. Fradet [12] gave 
another argument based on Reynolds' abstraction/parametricity theorem [29]. Fradet's formulation 
is closer to ours than Goldberg and Gloger's, since he represented the evaluation "stack" as a source- 
language term. However, none of these papers gives a complete formulation of the underlying 
dyn amic and static semantics of the language and thus the proofs of correctness are necessarily ad 
hoc. 

Finally, Purushothaman and Seaman [28, 32] and Launchbury [21] have proposed "natural" 
semantics for call-by-need (lazy) languages where the semantic objects include an explicit heap. 
This allows sharing and memoization of computations to be expressed in the semantics. More 
recently, Ariola et al. [5] (see also [4, 22]) have presented a purely syntactic theory of the call-by- 
need A-calculus that is largely compatible with our work. 

8 Summary 

Our paper provides a unifying semantic framework for a variety of garbage collection ideas includ- 
ing standard copying and mark-sweep collection, generational collection, tag-free collection, and 
inference-based collection. By making allocation and the heap explicit, we are able to reason about 
memory management using traditional A-calculus techniques. In particular, we are able to make 
strong connections between garbage collection and type theory. By abstracting away such low-level 
details as evaluation stacks, registers, and addresses, we are able to formulate complicated collec- 
tion algorithms in a compact manner and yet give a formal proof of correctness. The framework is 
reasonably robust, as demonstrated by the breadth of topics covered in this paper, and we expect 
that the framework can be extended to deal with other work on garbage collection. 
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